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ABSTRACT 

The Pell Grant Quality Control Study of 1982-83, the 
third stage of a contract with the Department of Education, was 
designed to identify program error rate, to measure the impact of 
increased validation activity, and to propose corrective actions to 
reduce the mi sallocat ion of program funds. A nationally 
representative sample of approximately 4,000 students was drawn from 
a stratified random sample of. 317 participating institutions. The 
results showed that Pell Grant recipients in 1982-83 were granted ' 
$129, or . 13 percent, more than they should have been. Both, student 
and institutional error dropped between 1980-81 and 1982-83. The 
study confirmed that institutions complied with the revised 
validation requirements' for the Pell Grant program in 1982-83, 
reflected in a $22 million reduction in the net Adjusted Gross Income 
error. Several correction action alternatives are presented to 
further reduce both student and institutional error. Stage One of 
this study determined program-wide rates of discrepancy between 
actual awards and what should have been awarded. Stage Two began the 
design of a quality control system for the Pell program, performed 
some error analyses, and prepared corrective action recommendations 
for specific features of related student* aid programs. Stage Three, a 
replication with significant improvements of Stage One, has the 
additional objective of determining changes in program error over 
time. Stage Three also includes an assessment of the degree to which 
institutions are fulfilling their responsibilities with regard to the 
expanded validation requirement. (Author/LB) 
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.INTRODUCTION 



THe PeirGrant Quality Control Study of 1982^83, the third stage of a 
contract with the Department of Education, is designed to identify 
program error irate, to rr^asure the impact of increased validation activity, 
and to propose corrective adtions that might further alleviate the reallo- 
cation of program, funds. 

A nationally representative sample of approximately *,000 students was 
drawn from a stratified random sample of 317 participating institutions. 
The upward trend in error between "1978-79 and 1980-81 has been turned 
around. However, findings suggest that'oveFall error in the Pell Grant, 
program remains high. Pell Grant recipients in 1982-83 were granted $129, 
or 13 percent, more than they should .have been. Both student and 
institutional error dropped between 1980-81 and 1982-83, Net overall error 
per recipient, net student error peF recipient, and net institutional error 
per recipient were down 2*, 9, and A9 percent respectively froVh the 1980- • 
81 levels. The conclusion is that one or morp of the elements of corrective 
action and regulatory change implemented by the Department of Education 
since 1980-81 have been successful in reducing but rvjt eliminating error, 

Tl^ study confirmed that institutions did a relatively good job of carrying 
out the revised validation requirements for the Pell Grant program in 1982- 
83. That work is reflected in a significant reduction (58 percent or $22 r 
million) in the amount of net AGI error when the application accurately 
represented dependency status and produced validation of income from the * 
proper person(s). Further, students selected fpr ^validation showed a 
markedly higher likelihood to. change their application in ways that would 
decrease their eligibility than did those not selected for validation. 
Savings in addition tc^ those measured by the . study accrue ^rom the 
deterrent effect of the threaFbf validation on the false filing of applica- 
tions by ineligible students. However, the data also show that the revised 
validation requirements, whi^did not adequately address cases where the 
dependency status was incorrect, failed to alleviate the 6 problem of 
improper self -classification of students as independent. 

Several corrective action alternatives are "presented*^ a means of further 
reducing program-wide error in general and student error in particular. 
These corrective actions are focused on reducing the $129 average error. 
Resulting savings and benefits can be reprogrammed to increase the^ 
maximum gr&nt size, to provide for more reasonable levels of administra-^ 
tive allowances, or to reduce the Federal deficit. 
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Background . A * . i 

Since its inception in the 1973-7* academic year, the Pell Grant Program— known 
• as the Basic Educational Opportunity Grant (BEOG) Program until 1981— has 
experienced significant growth in the number of students served and dollars expended, 
going from 185,000 recipients of $50 million in the initial year to an estimated 2.53 
million recipients and $2,425 billion in 1982-83. Along with that growth has come an 
pcreasing concern about the integrity of the program. Pell Grants represent a 
substantial portion of the Department of Education's budget, are received by a large 
number of participants, and thus haVe a high degree of visibility in both the. Congress 
and the Executive Branch. 

J> ., • * ' . 

Over the years, a number of steps have been taken by jhe Office of Student 
Financial Assistance to enhance the quality of Pell Grant delivery, including: ;, 

• Comprehensive computer edits of application data 

• Selective validation of application data by financial aid administrators 

• Data matches with other Federal sourqes of financial information* 

• . Expanded use of program reviews and financial audits 

,# Continued^ training opportunities for institutional ttnancicil aid and fiscal 
personnel A * ' * 

• Periodic redesign of the Pell Grant application form and instructions, 
• including extensive field testing ■ ' 4 

• Several studies of program administration ("targets of opportunity"), both 
internal and external to th$ Department of Education 

• » Increased awarehess of the importance of quality control among Depart- 

ment of Education employees 

• The inclusion of quality control requirement^ in contracted y/ork relating 
to the Pell Grant Program . 

• Expanded use of technology to improve both the speed and accuracy of 
processing. 

In September, 1980, the Office of Student Financial Assistance (OSFA) of the 
U.S. Department of Educatipn contracted with Advanced Technology, Inc., of McLefen 
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and Reston, Virginia, to continue ths 

year study to assess the accuracy and 

i 

recomnjsnd administrative changes to 




assurance initiatives with a threS- 
of the Pell Grant ' Program and 
Westat, Inc., of -Rackville, 

Maryland, has served as a subcontractor to Adva?JHE,Technology throughout the study.' 



During Stage , One of the study— the fifS^»' year of the contract (1980-81)— 
Advanced Technology and Westat examined a ^Hp nal sample of Pell Grant recipients 
' to determine eligibility and award calculation Westat drew the national sample 

of 4,500 recipients and interviewed them and their parents about their eligibility ancl 
financial situation, examining documentation at the same time. Advanced Technology 
staff visited the institutions attended by these students, ^xamined the records on Jhe 
students and any supporting documents on file, and Interviewed the financial' aid 
administrators. This same data collection procedure was followed in Stage Three—the 
third year of the contract (1982-83)— and is described below. , 

The .Stage One study determined program-wide^ rates of discrepancy between 

actual awards and what should have been awarded Recording to program rules and the 

documents examined. These discrepancies were then allocated to institutions, 

recipients or their parents, and application processors. On^'the basis of these 

discrepancy rates, the study identified error-prone groups of recipiehts. Finally, Stage, 

One suggested feasible corrective management activities to reduce error rates for 

every area in which error rates were excessive. 

i 

v 

During Stage Two (October, 1981 to December, 1982), Advanced Technology 
began tfte design of a quality control system for the Pell Program and performed some 
error analyses and prepared corrective action recommendations ioc specific features 
of related student aid programs. / - 

Stage Three of the study (1982-83) has essentially been a replication with 
significant improvements of Stage One, with the additional objective of determining 
changes in program error over time, especially changes potentially brought about by 
the expanded validation requirement* Stagd Three also included an assessment <Jf the 
degree to which institutions were fulfilling their responsibilities with regard to-that 
expanded validation requirement. 
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Copies of the three-volume, firial report for Stage Three wilL be available 
through Educational Resources Information Clearinghouse (ERIC) as was done for the 
thfee-volump, final report frorn Stage One. 4 
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• U METHODOLOGY 

* * 

♦ * 

'Sources of Data , 

* ^In jthe fall of I£S2, Advanced Technology^ and Westat selected 317 institutions 
participating in the Regular Disbursement System for inclusion irrtfoe study. A random 
sample of institutions, stratified by type and control and geographically clustered, was 
used for the study. The geographic cluster . technique was used to expedite data* 
collection and minimize study costs. An additional sample of students attending 
Alternate Disbursement System institutions was also selected on a simple random 
sample basis. Upon selection of the institutions, Advanced Technotogy field staff 
visited each institution during a dour week period And selected the student sample and 
assessed institutional compliance with new. validation requirements. 

control groups, one from among only sampled institutions and totaling about 
800 recipients^ and one from among all institutions and totaling about 10,000 
recipients, were drawn to test for experimental bias. The »first group was used to 
determine If ;the corrections behavior of unsampled recipients at sampled institutions 
was any different tf^an that of sampled recipients. The second group was used both to 
augment the first* group and to determine if s the corrections behavior of recipients at 
sampled institutions * was any different than -that 6i recipients at unsampled 
institutions. , 

The data. for Stage Three were collected from several sources, including: 

# . r 

Institutional Questionnaires 
Student Record Abstracts 

.Institution Control Group Forms ' 6 

Student Interviews ^ * 

Parent Interviews 1 



• . IRS Tak Returns v ■ <. ; ^ 

• ^ Financial Institution Records . ' / 

• ' Tax Assessor Records ) ' Vn 

• Computed Applicant Records of Student Aid Report 

Advanced Technology field staff collected -the first three documents during visits to 
each sampled institution during the spring of 1983. Westat interviewers collected the 
student and parent interview data during a comparable, but somewhat longer, period. 
' In addition, they collected the needed releases for the tax returns, financial institution 
records, aAd tax assessor records at the time of interview. The computed applicant 
records were obtained directly from the Pell Grant central processor. 

Data from all nine sources were cleaned through a series* of manual ajpd 
computer edits and "marginal" tabulations to identify illogical values. The seven files 
with student and parent data (all except the Institutional Questionnaire and the 
Institution Control Group) were then merged by a complex process to prepare for "best 
value 11 selection. After determining the most appropriate dependency status for each 
case, the computer program developed for this purpose selected the best available 
value for each data element used to determine the Pell Grant. "Best value" was 

^determined by a detailed set of priorities agreed to in advance. In each instance, the 
most highly documented source of that particular data element available to the study 
was assigned the highest priority. 

> * 

The best value selectibns were then used to. recompute the Student Aid Index for 
each case. That index was then used, in conjunction with the best values for cost of 
attendance, eligibility, and enrollment status, to compute the most correct Pell Grant 
award. That most correct award was then compared to the award amount paid or 
scheduled to be paid, with any difference being labeled as the p&^fcnent error for that 
particular case. 

-.••/., 

Mean error per recipient and mean error per recipient with error were then 
calculated for the sampled recipients and program-wide error was extrapolated from 
these mean values. All data were adjusted for nonresponse. 

Further analysis was performed to identify •specific sources oi error reflected in 
C_ 2 - 2 Jf^Aj&ncat 



the aggregate data. Specifically, program errbf was categorized into: 



• * Student error 

• * Institutional error ; ' 

\ \ 

Each of these was. in turn decomposed into types or sources of error-that contribute to 
the total. Program-wide estimates of error attributable to the various types and 
sources of error were also made. 

r v 

A more detailed description bf the methodology, used is found in Quality in the 
Pell Grant Delivery System: Volume 3, Procedures and Methods . 

Limitations * 

• • *t 

' ' f> . 

There will undoubtedly be a great temptation to attempt direct comparisons 
between the results of^fche 1982-83 Pell Grant Study and the findings bf studies which 
preceded it, especially the 1980-81 assessment it was designed to replicate. Such 
comparisons* shoulcf^be done with extreme caution, however, for a number of reasons.^. 

The first limitation to comparison is that the program legislation/regylations 
underwent some changes between the two years under study. Those aspects of change 
that could, and probably did, influence the results to some unknown degree are as 
follows: - 

» * 

• Social Security and VA Educational Benefits becafhe, for 1 982-^83 only, a 
direct adjustment to the Pell award, rather than being used as elements in 

* the Student Aid (formerly Eligibility) Index computations. 

• • For 1982-83, dependent student estimatednncome was used instead of the 
previous year's income when the estimate was less than 60 percent of the 
previous year's reported income. 

• The maximum award for each 1982-83 was $1,800, compared to $1,750 in 
1980-81, thus expanding the potential impact of error in maximum grant 
cases by $50 each. : * \ 

• Th^ Student Aid Index computation formula was changed with regard to 
Standard 'Parental Contribution from Income, moving from a flat 10.5 
percent "taxation" rate in 1980-81 to . a progressive "taxation" rate 

( ' ' ' 
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^ • v ranging from 1 1 to' 25 percent in 1982-83; this increased the impact of 
misreportecf income on the Aid Index and, consequently, on th? Pell Grant 
award amount determined to be in error. 

• Institutions were permitted td use average costs for tuition, fees, arwJ 
^ 9 institution-provided room and board, rather than the actual charges 

' required in 1980-81. . » 

• The Statemeht ,of Educational Purpose (SEP) was printed directly on the 
institutional copy of the Student Aid Report, to lessen the likelihood of 
overlooking the submission of that required document. 

The second limitation to comparison results from continued expansion of the 
program funding and participation. All- aggregate dollar differences must be viewed 
carefully because there is a different base for the two yearsL For example, program- 
wide estimates of error for 1982-83 are based upon 2.53 million Pell recipients, 7 
percent more than the 2.36 million in the 1980-81 stu^r. Thus, the $650 million 
estimated absolute case 'error (the sum of the absolute values of both overawards and 
underawarcjs of more than ^$2 per case) for 1980-81 would have beerv approximately 
$695 million had the. number of Recipients in 1980 T 81 been comparable to the 1982-83 
participant level. For this reason, mean error per recipient and mean error per 
recipient with error, along with the percent of cases with error, are the rrtost 
meaningful and accurate bases for comparisons with results of previous studies. 

The third limitation is the study design and methodology used. Although there 
are many similarities in methodology, especially between 1980^81 and 1982-83, there 
were some important enhancements^ made for 1982-83. More experienced and 
knowledgeable persoWiel were used to abstract student records at institutions, using an 
expanded data collection instnhnent. More effort was made to obtain official copies 
of IRS returns, especially from The parents of applicants filing as independent 
students. Student and parent interviewers were more experienced (many were 
repeaters)' and presumably more adept at Jferreting out confirming-or conflicting data 
during the course of the interview; More sensitive best value selection methods were 
used. Thus, we were probably more successful in identifying discrepancies in 1982-83 
than ifi any previous study. 

Finally^teomparability is limited by environmental changes beyond the boundaries 
^of the Pell Grant program or the quality control study. For example, tax law changes 
between, the two years created an exclusion for* interest income in 1981, which made it 
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. more likely ^for applicants *o underreport nontaxable income on their 1982-83 
application. The deductions in Spcial Security Educational benefits, and applicant 

♦ uncertainities as to what their reduced benefits would be, may have caused jnore 
applicants to overstate their assistance from this source than they would have 
otherwise. Additionally, there may have been any number of economic conditions that 

caused applicants to behave differently icbl9^2-83 than in 1980-81. 

* 

These points do not make the 1982-83 results any less meaningful, but they do 
. suggest the need for caution in interpreting any comparison of those results with the 
results of previous studies. \ 



One additional point should be considered in interpreting the findings. Estimates 
of error are based on data collected during the spring. There is the possibility that 
institutions will correct some errors during the normal end-of-the-year self-correction / 
(reconciliation of accounts) process. However, the 1980-81 BEOG Quality Control 
Study suggested that the net effect of these self-corrections on estimated program- 
wide error is minimal. ^ J 
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FINDINGS 

m ■ 

The findings of the 1982-83 study will be considered in three partes, overall error) 
student error, and institutional error. For purposes of this study, error has been 
defined as it was in 1980-81, i.e., a discrepancy of more than two dollars (in either 
direction) between the award amount paid of scheduled to >e paid at the time of data 
collection and the award amount computed with the' best values described in * the 
methodology section. The Glossary of Terms provided in the Appendix defines all 
error terms used in this study. / J - , . * 

Overall Error • 

The following^statements summarize the findings related to overall program- 
wide estimate of error iri the 1982-83 Pell Grant Program. 

' .-" • . p . 

• Absolute case - error (adding underawards and overawards together) is . 
estimated to. be $605 million, or 25 percent of the dollars awarded in the 
program. 1 . ; . 

• Absolute ^ase error averages $23$ per recipient. ' , 1 

• Net case . error (allowing overawards to be offset by underawards) is 
estimated to be $526 million, or 13 percent of the dollars awarded. 

• Net case error averages $129 per recipient. 

• About 63 percent of alf eases have award erro*J?eyond the $2toferance 
level. y ; ^ 

• Absolute total error, which does not allow student and institutional errors 
in the samescase to off set each other, is $6*9 million, or 27 percent of 
total program expenditures. y ' a 

• • • . - ■ *. 

• Net total error, wh^re student and institutional error can offset each 
other, is $316 million, or, like net case error, 13 percent of program 

r e expenditures^ " . r ^ ^ 

• Total (freraward error is estimated to be^ $*82 million, while total 

underaward error is' $166 million. • J . 

Aw 

' * „ ' - 
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• Case overaward error is estimated to be* $*65 million* while case 
underaward error is $139 million, 

•These and other overall error findings are presented in Table 3-1, on the Total and 
Case rows of each grid. 
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Student Error " ^ 

> 

In developing corrective actions, it is important to identify the source of the 

error. The student component of overall error is that jvhich is attributable to 

* * * 

discrepancies in application values submitted by students and/or their parent(s). Key 

findings in this area are the following: / ' , 

* , * 

• The program-wide estimate of student error is $328 million, composed of 
$272 million in overawards and $55 million in underawards. 

• Absolute student e^ror per recipient ^averages $129. 

t The program-wide estimate of net student errbr is $217 million. 

• ' Net student error pier recipient averages $86. 

• Student error is present in 39;* percent of all recipients, 30.6 percent of 
. the time involving an overaward, and 8.8. percent of the time involving an 

underaward. 

To be most meaningful, student error neecis to be further decomposed to identify 
the types of stujdfcnt error that are most prevelant. The following points highlight the 
results presented in Table 3-^2 regarding the types of student error that were 
identified. 



The most costly student error type is attributable tp improper income and 
asset reporting associated with error in dependency status. This 
Dependency Status, error totaled an estimated $6* million of net *err9r. 
Most of these casgf, which were of low frequency but high dollar impact, 
were students coming to be independent when they did not, based upon 
best values, quaftfy. Only a small proportion were independent students 
clainr)ing to be dependent. 

Other Nontaxable Income discrepancy is the second most costly item of net 
student error, totaling $*6 million. Of that total, unemployment 
• compensation, child support, other welfare benefits, and excluded interest 
Sare the , most „ significant contributors, in that order. .Excluded interest 



TABLE 3-1 

A SUMMARY OF AMOUNTS AND RATES OF ERROR 



ABSOLUTE ERROR 





Promm-Vide Estimate 


Mean 
Error per 
Recipient 

w) 


Cases 


Mean 
Error per 
Recipient 
V/Error 




J 

/ Program-Vide Estimate 
<$lfclilons) a£t*J> 


Mean 
Error pes; 
Recipient 
1 C$) 


Cases 
V/Error* 
(%) . 


Mean 
Error per 
Recipient 
V/Error 
($) 


Error 


($ Millions) 


A^artleaV 


V/Error* 


Error 


Institutional 




. 13 


127 


33.5 


379- 


Institutional 


1 

99 


• 


39 


33.3 


117 


Student 


32* 


, 1* 


129 


39.4 


32$ 


Student. 


217 


9 


86 ~ 


39.* 


217 


Tojal * 


6*9 


27 


256 


62.7 


•08 


Total 


316 , 


13 


125 


62.7 


199 


Case 
' 1 


605 


25 


239 


62.7 


381 


Case 


326 


13 


129 


62.7 


205 


























t 




OVERAWARD ERROR 








UNDERAVARD ERROR 








Program- Vide Estimate 


Mean 
Error per 


Cases 


Mean 
Error per 
Recipient 




Program-Vide Estimate 


Mean 
Error per 
^ Redolent 


Cases 


Mean 
Error per 
Recipient 
■V/Error 
($) 


Error 


($ Millions) 


(% of $ 
Awarded)* 


Recipient 


V/Error* 
<*> 


V/Error 
($) 


Error 


<$ Millions) 


A&S& 


V/Error* 
(%) 












Institutional 


210 


9 


83 


15.7 


528 


Institutional 


-111 


5 


-*• 

r 


17.8 


-2*7 


Student 


272 


11 


108 

o 


30.6 


351 


Student 


-55 


2 


-22 


8.8 


-2*9 


Total 


*82 


20 


190 


„v 


•59 


Total 


-166 


7 


-66 


21.2 


-309 


Case 


465 


19 


18* 


•1.3 


••• 


Case ^ 


-139 


6 


-55 


21.2 


-259 



\ 



I 



* Amount of Pell awards is $2.* billion for 1982-83. 

b Error is defined as a discrepancy of plus or minus $2 from the best award. 
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TABLE 3-2 

STUDENT ERROR BY APPLICATION ITEMS 



Program- Wide Estimate of 
Net Award Error 



Application tfjerh 


I ($ Millions) 

«» 


Rank 


Adjusted Gross Income/De^ndency Status* 


80 


- 


Adjusted Gross Income 


16 


7. 


Dependency Status** 


6* 


1 


Other Nontaxable Income 




2 


Household Size . 


3* 


3 


Number in Postsecondary £*Jucation 


23 

i 




Home Equity 


18 


5 


Dependent Student's (and Souse's) Assets 


17 


6 


Dependent Student's (and Souse's) Income 


12 


/ 8 


U,>. Taxes Paid 


2 


9 


Spouse's/Mother's -Earned ty^ome 


2 


10 - 


Re^I Estate/Investment Equity 


2 


11 


AFDC/ ADC ^ 


. 2 


12 


VA Educational Benefits 


1 


13* 


Marital Status (Parent) 


1 


1* 


Business/Farm Equity 1 


1 


15 


Cash/Savings/ Checking 


1 


16 


Medical/Dental Expenses 


+ 0 


17 


Applicant's/Father's Earned Income 


- 0 


18 


EJerhentary/Secondary Tuition 


- 1 


19 


Marital Status (Student) 




20 


Social Security Benefits (P^eht) 


- 3 


21 


Social Security Educational benefits 


- 17 


22 



ERKT 



*In 1980 - 1981 Adjust^ Gross Income/Dependency Status Error were presented 
together as Adjusted Qfoss Income (AGI) Error with an explanatory footnote. The 
rationale for this w^ that Dependency Status Error represented use of the 
incorrect AGI and otlyr application values. In the interest of a greater clarity, 
we are presenting AGl Error and Dependency Status Error separately, since they 
are, in fact, errors that represent, different problems. For purposes of 
comparison, we have $P*Wn the combined AGI/Dependency Status Error for 1982 - 
1983. v 

** Dependency status errtfr is computed in the same way al overall student error (an 
approach which is mo^ accurate but cannot be used with individual application 
items). The figure reported here treats dependency status switchers with 
SEP/FAT/Invalid SAR error as ineligible. If they were considered eligible, 
dependency status swifter error would be $70 million. 

3-* 
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is the most frequent discrepancy, but averages less than $180 per case. 
Misreported child support, on the other hand, averages about $2,%00 per 
case. 

The third most costly discrepancy is Household Size ($3* million), followed 
by Number in Postsecohdary Education ($23 million), and Home Equity ($18 
million). 

Misreporting of Dependent Student's (and Spouse's) Assets accounts for $17 
million in net student error. 

Misreporting of Adjusted Gross Income for cases not involving an error in 
dependency status accounts for an estimated $16 million, which puts that 
type of error in seventh place. 

The only other net student error exceeding $10 million of program cost is 
Dependent Student's (and Spouse's) Income, which is estimated to be $12 
million in error. 



Institutional Error 



c 



Institutional error is that which is attributable to discrepancies in determining 
student general eligibility, enrollment status, and/or cost of attendance, or an error in 
calculating the award amount (including misreading of the payment schedule) or 
making an improper disbursement. Table 3-1 shows the aggregate institutional error 
findings, while Table 3-3 summarizes the institutional error components. The 
following points highlight the institutional error which was identified. & 



Absolute institutional error is estimated to be $321 million, or 
approximately 13 percent of total program expenditures. 

The average absolute institutional error per recipient is $127. 

Net institutional error program-wide is $99 million, or about * percent of 
total expenditures. 

Net institutional error averages $39 per recipient. 

Approximately $210 million of the institutional error is in the form of 
overawards, while underawarcjs^account for an estimated $111 million of 
the total. K 

Institutional overawards constitute 15.7 percent of the cases, whereas 17.8 
percent of the cases are underawards; however, the average amount of the 
overaward error is more than twice that of the average underaward. 
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TABLE 2-3 

COMPONENTS OF INSTITUTIONAL ERROR 



I 







ABSOLUTE ERROR 






1 . . • 


NET ERROR 




1 . 


Mean Error 
per Recipient 
($) 


Mean Error 
per Recipient 
W/Error 
($) 


Profcram- 

#Ide 
Estimate 
($ Millions) 


Cases 
W/Error 
(%) 


I 

Mean&ror. 
per R**tfpient 

(ft 


Mean Error 
per Recipient 
W/Error 

($) 


Program* 

Wide 
Estimate 
($ Millions) 


Edibility Error 


56 


l f 07S 




5,2 


i H 


1,078 


1*2 


Insufficient Program Length 




79 ' 




.03 


* - 


79 


•1 V 


Nondegree Program 




1,051 




.*9 




1.051 


13 


Not Parent Institution 


\ 2 


1,018 






4 4 


1*018 


5 


Default on Loan 


* 


505 


> 


•03 




505 


- A 


Less Than Half Time 


« 


601 


1 


•09 




601 


1 


Unsatisfactory Academic Progress 


2 


656 


e 


• 38 




656 


6 


Invalid SAR 


3 


855 


a 


•36 




855 


8 


No Statement of Educational Purpose 

* 


4 


1,043 


10 


•39 




1,0*3 


10 


\ No Financial Aid Transcript 


37 


1,168 


9i 


3.2 


M . 


1,168 


95 


No SAR in File 


2 


1,725 




• 10 




1,725 


% 


Has Bachelor's .Dcw^^** 
















Not a Citizen or Eligible Noncitizen** 








/ 








Disbursement Error 


77 


255 


m 






-50 


-38 


Cost of Attendance Error 


19 


180 


»* 


10.8 




-76 


-21 


Enrollment Status Error 


58 


260 




22,3 




-71 


-39 


Calculation/ Accounting. Error 


22 


182 


» . 


12.3 




77 


2% 



•Less than $!• ( 

••There were no instances of these errors in the sample used; 



18 



Advanced 
Technology 



ERLC 



• The most prevalent form of institutional discrepancy is Enrollment Status 
error, estimated to constitute $1*7 million of absolute program-wide error 
and more than 22 percent of the cases. 

• Calculation and accounting discrepancies are the second most frequent 
occurrences, representing slightly more than 12 percent of thp cases and an 
estimated $57 million of absolute program-wide error. 

• Cosj of Attendance error accounts for about $49 million of absolute 
program-wids error and about 11 percent of the cases. 

• The only oth$r type of institutional error that involves ms>re than 1 percent 
of the cases with error is the absence of the Financial Aid Transcript, 
which occurs in 3.2 percent of the cases. If each of these awards is 
considered to fee entirely in error, the program-wide cost is estimated to be 
$95 million, of the second most expensive institutional error found. 

• Other than F/VT error, very little discrepancy based. upon general eligibility 
requirements tyas found. 

It seems advisable tQ make a distinction concerning overall institutional error 
relatedlto the absence of the Statement of ^Educational Purpose (SEP), the Financial 
Aid Transcript (FAT), or # valid Student Aid Report (SAR) in the # institution's record 
system. Thifs far, all discission has considered such cases to have the total amount of 
the "award in error. Cl^rly, the failure to have a valid SAR or SEP on file is a 
violation of program requirements. However, it is a procedural error that may not 
make the amount of the ^ward incorrect. The error associated with the absence of a 
FAT for a transfer student is more difficult to judge. First, disbursements may be 
made in the first payment period without the^ FAT in hand, so considering the entire 
award in error (as we h$ve done) overstates the case. Second, a high percentage of 
recipients would maintain their eligibility for tfte full amount of their otherwise 
properly calculated ayard if the FAT were to be received within that academic year. 
The only awards that wouW be potentiallyvin error if the FAT were received would be 
those made to mid-year transfers who had previously used some of their 1982-83 
entitlement at their previous institution. All ojher FAT error cases would have 
legitimate awards upon presentation of the required transcript. 

For these reasons, we have provided an additional table wtiich excludes 
SEP/FAT/In valid SAR errc?r cases. Table 3-1, which has already been presented, tends 
to overstate institutional %rror by an amount equal to at least the disbursements made 
in the first payment period for transfer students. Table 3-4, which presents the same 
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A SUMMARY OP AMOUNTS AND RATES OP ERROR 
TREATING CASES WITH $EP/F AT/INVALID SAR ERRORS AS ELIGIBLE 



Mean 

Mean Error per 
Program-Wide Estimate Error oer Cases Recipient 
(%ofi Recipient W/Error* W/Error 
Error ($ Millions) Awarded)* ($) (%) ($) 


Mean 

Mean. Error per 
Program- Vide Estimate Error oer Cases Recioient 
' (%of$ Recipient W/Error* W/Error 
Error ($MilEonsV Awarded^ <$) (%) Q 


Institutional 217 9 86 30.6 2S1 

Student 3%3 1% 137 41.1 332 

/ 

Total -362 -23 223 61.1 36% 
Case 317 2l 20%. 61^ 33%^ 


Institutional -13 1 .3 -3 , 30.6 -17 

* 

Student 231 10 91 %1.1 222 

total 217 9 8% 61.1 1%1 

• • i • t 

; Case 226 9 , 89 1%6 



/ 



OVERAWARD ERROR 



UNDERAWARD ERROR 



Error 



Program-Wide Estimate 
($ Millions) AwdteoV 



Mean 

Mean Error per 

Error per Cases Recipient 
Recipient W/Error 0 W/Error 
($) (%) ($) 



Error 



Program-Wide Estimate 



($ Millions) Ai 



Mean 

Error per 
Error per Cases Recipient 
Recipient W/Error* W/Error. 
<$) <%) ($) 



Institutional 


102 


% 


%0 


12.2 


331 


Institutional 


-113 


3 


-%6 


Student 


288 


12 


11% 


32.0 


333 > 


: Student 


-37 


2 


-23 


Total > 


390 


16 


*> 

13% 


39.1 


39% 


Total 


-173 


7 


- -68 


Case 


371 


13 


1%7 


39.1 


373 


Case ' 


-1%3 


6 


-37 



18.% 
9.1 

22.0 

i . x 

22.0 



-2%8 

-230 
O10 
-261 



* Amount of Pell awards is $2.% billion for 1982-83. 

b Error is defined as a discrepancy of plus or minus $2 from the best award. 
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information founds^in Table 3-1 excluding cases with SEP/FAT/Invalid SAR error, 
^understates the, error : to the extent that <some (probably few) cases without an FAT 
would prove to have, upon receipt of an FAT, an incorrect award amount. 

One other point of clarification should be made Regarding these findings. The 
tolerance level used to define error, $2 over or under the correct award, is recognized 
as quite stringent. It is used to enhance comparability with prior findings. However, 
we have computed alternative amounts and rates ©terror at various tolerance levels. 
While the percentage of cases with error is reduced substantially by increasing the 
tolerance level, estimated absolute and net dollar error do not change significantly. 
These alternate figures are shown in Table 3-5, which follows. 

l 




ERIC 7 " 23 



TABLE 3-5 



AMOUNTS AND RATES OF ERROR WITH SELECTED 
TOLERANCE LEVELS OF NO ERROR* 



Program-Vide Estimate 



Absolute 
($ Millions) 



Net 
($ Millions) 



Estimated Cases 
with Error 
(%) 



Case Error ($)^ 

+ 2 

± 25 

- +--50- " 
7 100 



605 
598 
591 
577 



326 
320 
31 8 
316 



62.7 
55.8 
49.7 
42.4 



Student Error ($) 



+ 2 
+" 25 
+ , 50 
+ 100 



328 
323 
318 
307 



217 
212 
209 
207 



39.4 
36.0 
30.9 
25.8 



Institutional Error ($) 



+ 2 
+ 25 
+ 50 
+ 100 



321 
318 
315 
307 



99 
98 
99 
100 



33.5 
,27.5 
24.9 
20.6 
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EFFECTS OF VALIDATION 

> 

"beginning in the 1982-83 academic year, OSFA instituted new requirements for 
institutional validation of Pell Grant applicants. The proportion of students selected 
for validation rose from less than 10 percent, mostly chosen on the basis of pre- 
f established criteria or PECs, to more than 60 percent, including a period of time when 
all apparently eligible applicants were selected. At the same time, there was a change 
in emphasis of the validation. For most validated students, the institution was 
required only to verify ^ limited dumber of application values with a copy of the tax 
return for the previous year (1981) for independent students or . the parents of 
dependent students. For those not filing a tax return, a signed statement of non-filing 
was required. 

Completely estimating the savings attributable to validation is not possible with 
existing data. Savings accrue for several reasons: * ' 

■ * * 

• Ineligible students, knowing that they may be validated, do not apply for a 
Pell Grant 

• Ineligible applicants do not alter their applications, in order to lower their 
SAI and felsely appear to be eligible 

• Eligible applicants do not falsely alter their 'applications in order to' 
^ increase their eligibility. 

The data base only allows us to measure the second two components, thus the 
following magnitudes should be considered as lower-bound estimates. 

The data from the 1980-81 study projected a substantial $101 million program- 
wide error attributed to use' of incorrect Adjusted Gross Income : (AGI) in the 
determination of the Student Eligibility (now' Aid) Index. It was noted in the Final * 
Report that this estimate of error consisted of two types, cases where the incorrect 
AGI for the correct individual was reported on the application, and cases involving the 



use of erroneous AGI (and other application items) because of incorrect selection of 
the proper dependency status category, * 

To^ehieve greater clarity for the 1 982-82 tiata, if was decided to separate these 
two components of error that were reported jointly in 1980-81. This better identifies 
the possible sources of the error arid . provides a more precise indication of the 
effectiveness of validation. By separating' misrepor.ted AGI for- cases with proper 
dependency status from thoSe with improper dependency status, an<3 reconstructing the 
same decomposition for 1980-81, the following results were identified: 



1980-81 



Combined AGI/Dependency 
Status Error 

AGI Error (correct dependency 
status selected) 



$101 million 



38 million 



1982-83 



$ 80 million 
16 million 



Dependency Status Error 
(causing wrong AGI to be 
reported) 



63 million 



6* million 



Thus; while validation apparently reduced, by 58 percent, AGI error where the correct 
dependency status was involved, the Dependency Status error was relatively unaffect- 
ed by the expanded validation requirements. Because the validation procedures for 
independent students (who constituted 93 percent of erroneous dependency status 
cases for 1982-83) required collection of only<the independent applicants tax return, 
the inappropriateness of a claim of independent status, would likely not be discovered 
by validation. Had the parents' tax return been' requested, as it was in the 1982-83 
study procedures, some 33 percent of the dependency statu$ errors would have 
probably been discovered and corrected. The balance of such errors appear to result 
from, misreporting of support received from parents, or time residing with parents, 
which are likely not to be discovered by review of IRS forms when the student was not 
claimed as an exemption. ■ 

The value of validation in detecting errors other than Dependency Status, 
however, was confirmed. Key findings are: * 

\ 
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Net AGI error is 81 percent lower among students selected for Validation' 
and validated compared to' those neither selected nor validated . 

• The vast majority of institutions collected c the required verifying documen- 
tation for their students who were H lagged" for validation by ED, The 

Federal tax returrt was the predominant form of documentation ■ 

■« ■ * • , ■* 

■ • ' ^ - Validated students were more likely toTfeVise their Adjusted Gross Income 
(AGI) and, Federal ; taxes paid initial application data than were non- 
validated students." Further, those selected for validatipft and making 
k : : revisions were more likely to raise their AGI and/reduce th^ir ' taxes paid- 
revisions that tend to lower eiigibility T -than weref those not selected for 
validation , - ^ 

# Among those recipients who made corrections to their application, 
percent of those /randomly selected for validation made changes that 
decreased their * eligibility wliile only 18 percent made changes that 
increased their eligibility. In comparison,: of those not selected for 

■. „ validation only 9 percen.t made changes that decreased their eligibility 
., , . while percent made changes that increased their eligibility. 

* ■■■ " . : 

This* change in correction, behavior, which is counter to the self-interest o{ the 

applicants, can be attributed primarily to validation. " 

The changed, requirements and increased number of studervfPto b6 validated in 
1932-83 didyresult in som'e problems however. Household size, number in college y jand 
other nontaxable income showed increases in net error, per recipient^ bet weeing 980-81 
and J 982-85 of 1 percent,/63 percent, and 11* percent, respectively. These items Wece. 
* required td be, valdiatfed for : those selected Jor validation ip 198Q-81, but not in 1982- 
83. Also, while the fnajorLty of institutions collected the required 'verifying documents 
ih 1982-83/ most believed that they were unduly, burdened by the changed nature of the 
process. The -reasons cited most often were del£yvextr£ work, pn confusion due to 
"the late .arrival of the Validation'. Handbook, difficulty, in verifying. 5te^al Security and 
Veterafts. Adnrfirvj^tration benefits, and difficulty : in obtaining . do^'mentation from 
student^ .\ " \.- t : '.- .r.V- " : ' .V>'' \ 
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RECOMMENDED CORRECTIVE ACTIONS 



The iriajor thrust of any quality control study should be the development of 
corrective actions. These corrective actions should be directed toward economies 
associated with those few items that cause the most error ("the vital f ew") rather than 
the less optimal approach which attempts to cover the "trivial many" souces of error. 

Volume 2 of the Final Report reviews the role of corrective actions within' the 
context of a quality control effort, evaluates the apparent effectiveness of prior 
corrective actions taken by the Department, and provides a series of recommendations 
for reducing specif ic application and institutional processing errors, as well as general 
, delivery system enhancements. Ten specific error types, which account for a vast 
majority of the student and institutional errors identified by the study, were the focus 
of the recommendations. More broadly-based recommendations for improvement were 
directfed^at keeping error out of the system through simplifying and improving the 
application process and through formalizing; the role of the institution as the focus of 
quality control. The recommendations advanced in Volume 2 are summarized in the 
following sections. 

SPECIFIC ERROR CORRECTIVE ACTIONS 

Our specific recommendations are of two types, one to reduce application or 
* student error, and another to reduce institutional error. 

Application or Student Error 

In addressing the six application error areas, two themes were followed: 

• r ' ■ * 

• . pargeting validation toward the types of students (e.g., first^qrie indepen- 
dent srtudents under 20 years of age) or types of error (e.gJpiftcome) that 
would have.^the highest payoff. As discussed in Section i.l, this is a form 
1 „ 'of remedial corrective actions. 

* - . • ;.. . 
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• Improving the application form by either clarifying definitions (e.g.,, 
ndntaxable income) or requesting clarifying information (e.g., names of 

1 ¥ household members). This is a form of preventative corrective actions. 

Six recommendations address errors made by applicants in completing their 
apjjUcations for Pell Grant assistance. 

•» 

• Reduce the improper applicant determination of dependency status by 
expanded validation of first-time independent applicants and more strin- 
gent documentation requirements for those validated. 

• Enhance the accuracy of reporting other income and benefits through 
expanded itemization of the several components of this application item to 
reduce accidental * omissions and modified validation selection criteria to 
identify cases where other income sources are suggested by reported data. 

• Maximize the accurate reporting of household size by requiring that 
applicants list by name, age, and relationship the individuals being included 
in household size. The Department should also consider replacing house- 
hold size with number of tax exemptions, 

• Improve the accuracy in reporting the number of family members in 
postsecondary education by including the institution to be attended by the 
aforementioned list of family members and clarifying the instructions as to 
whether the parents of dependent students who may be attending are to be 
included in the reported number. 

• Continue to improve upon the accurate reporting of AGI by encouraging, 
but not requiring, the completion of the Federal tax return before 

* completion of the Pell application and using more refined validation 
selection criteHa to focus upon the most error-prone applicants. 

• * Maximize the accuracy of reported dependent student income by validating 
each applicant profe cfting a drop exceeding 50 percent in income from the 
base year to deter minSThe reasonableness of the projection. 

• <r - 

Institutional Error 

In addressing the four areas of high institutional error, two other themes were 
followed: 

, . ( 

• Simplifying procedures and policies, for example, of enrollmen.t;;?tatus 
regulations and the pay menl; schedule ^&<Kt'& r 

• Pitting more emphasis on the institution as the focus of quality control* 
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Four recommendations are aimed at tl^e redaction of institutional processing 
error. 



• Investigate the feasibility of a date-certain for the determination of Pell 
recipient enrollment status, with the institution "held harmless" for 
changes of status after that date each enrollment period, as a means of 
reducing the occurrence of erroneous status" projections without confirma- 
tion, A 

• Institute a more structured cost ot attendance formula that reduces the 
amount of variation occurring from mid-term changes in credit hour load 
and room and board arrangements, and investigate the desirability of 
establishing a date-certain for determining the cost of attendance to^be 
used for a given enrollment period. r " N A 

• Develop an individualized payment schedule for each official Student Aid 
Index, to be incorporated on the Student Aid Report, a^ a means of further 
reducing the misreading of the current payment schedule during institu- 

, tion^} processing. 

• Consider eliminating the current requirement for obtaining a Financial Aid 
Transcript (FAT) from each transfer Pell recipient except those transfering 
during an award year, as a means of reducing the error resulting from the 
unavailability of this document in institutional files; alternatively, consider 
allowing a "proxy 1 ! FAT to document reasonable attempts to Secure a 
hardcopy transcript. 

Implementation Costs and Burdens 

All of these recommendations can be implemented by either ED^administrative 
action or regulatory revisipn. Most of the proposed corrective actions involve a 
decreased burden or little or no change in the current requirements for all partici- 
pants. 

FURTHER IMPROVEMENTS IN THE PELL GRANT DELIVERY SYSTEM 

In addition to the preceding recommendations to address specific types of errors 
identified by this study, most of which could be considered short-term actions with 
minimal implementation burdens, a number of more broadly based and potentially 
longer range recommendations for improving the overall delivery system were also 
developed. These proposals $iso follow from the two sets of themes used to develop 
the specific recommendations for application and institutional errors. Furthermore, 
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these broad-based recommendations can be organized around th^/following more 
general themes: 

• Keeping error out of the system through simplifying and improving the 
application process 

• Formalizing the role of the institution as the focus of quality control. 

Recommendations for Keeping Error Out of the Application Process 

In this arena, there were five prospective actions identified as having potential 
for improving the accuracy and efficiency of the delivery of Pell Grants to eligible 
students. These recommended enhancements were: 

• Overall simplification of the application form to reduce confusion on the 
part of the applicant, perhaps through a technique already field-tested by 
the Department and known as the "split form," a separate form and 
instructions for dependent and independent applicants in a common packet. 

• Modification and possible redefinition of dependency status, primarily 
through substituting criteria which ascertain whether the applicant has 
provided or will provide his or her own residence and was or will be self- 
supporting, rather than the current criterion of not residing with a parent 
or being supported by parental resources. 

• Simplification of the Family Contribution Schedule through the deletion 
from the Pell formula of financial data elements which are difficult for 
applicants to provide easily and accurately, difficult to document, and 
contribute little to the reliability, sensitivity, and equity of the formula. 

• Establishment of a firm cut-off date for student-initiated corrections and 
the screening of ^those corrections by a financial aid administrator to 
reduce resubmission of data intended to artificially increase eligibility. 

• Prohibition of corrections subsequent to completion of validation of 
previously submitted data. 

Formalization of the Role of the Institution in Quality Control 

In addition to the specific recommendations regarding simplification of the 
application process, an administrative action that formalizes the institution as\the 
focal point for quality control is needed. Several steps are desirable if this 
responsibility is to be placed clearly upon the institutions, including: 

x / 



• Quality control regulations that provide, the necessary authority for 'dele- 
" gating such responsibilities to institutions and require a minimal quality 

control plan to be developed and implemented. 

• Incentives for institutions to reduce error and maintain it at an acceptably 
low level, perhaps through a> variable administrative expense allowance. 

• Substantial levels of technical assistance in the form of a quality control 
handbook, inclusion of quality control modules in training programs, and 
the availability of quality control specialists to assist schools in developing 
and implementing effective quality control plans. V 



These further refinements in the delivery system, in conjunction with the error- 
specific recommendations advanced in the Final Report, should reduce the level of^ 
student and institutional error in the Pell Grant program. The financial aid community 
working as the focal point of quality control should be able to bring about improve- 
ments in quality as has been done in other Federal entitlement programs. The AFDC 
error rate has fallen to 7.3 percent from 9.* percent between 1978 and 1982. For the 
1981-82 program year the national overpayment error rate for the food stamps 
program was 9.8 percent. Finally, the Medicaid error rate has been reduced to 3 
percent from nearly 7 percent during the 1978 to 1982 time period. 
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APPENDIX # 
' GLOSSARY OF TERMS* 

The use of error terms has been standardized throughout this document. The 
definitions of these terms are as follows: 



^Overaward 
Underaward' 
Student error 
Institutional error 

Case error* 

Case overaward 
Case under award 
Absolute case error 

Net case error 
Total error 

Total overaward 
Total underaward 
Absolute total error 

Net total error 



amount of award that is greater than its calculation based 
on verified values would indicate * 

amount ctf award that is less than its calculation based on 
verified data would indicate 

error attributable to discrepancy in application value(s) by 
the student or th£ student's parent(s) 
i 

error attributable to 1) discrepancy by institution in 
determining student's eligibility, enrollment status, and/or 
cost of attendance and/or 2) discrepancy by institution in 
calculating or disbursing^ward 

error in award per recipient, regardless of the source to 
which it is attributable (See absolute case error and net 
case error below for further distinctions.) 

overaward per recipient or case, regardless of its source 

underaward per recipient or case, regardless of its source 

case overawards plus the absolute value of case 
underawards 

case overawards minus case underawards 

error attributable to students and institutions (See 
absolute total error and net total error below for further 
distinctions.) . 

Sum of institutional and student overawards 

sum of institutional and student underawards ^ 

sum of institutional and student error for overawards plus 
the absolute value of the sum of institutional and student 
error for underawards 

sum of institutional and student error for overawards 
minus the sum of institutional and student error for' 
underawards " 



*Case error is usually less than absolute total^ error because of ' overlapping 
contributions of students and institutions to error or because underawards attributed 
to one source may be offset by overawards to another. 



